FINAL  REPORT 


Final  Report  at  Camp  Sibert  -  Gadsden,  Alabama 


ESTCP  Project  MM-0437 


NOVEMBER  2008 


Dr.  Erika  Gasperikova 

Lawerence  Berkeley  National  Laboratory 


Approved  for  publie  release;  distribution 
unlimited. 


En\  ironmcntal  Sccurih  Tcclinology 
Certification  Program 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

NOV  2008  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

Final  Report  at  Camp  Sibert  -  Gadsden,  Alabama 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Lawrence  Berkeley  National  Laboratory,!  Cyclotron  Road 
, Berkeley, CA, 94720 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  aS 

unclassified  unclassified  unclassified  Report  (SAR) 

45 

Standard  Form  298  (Rev.  8-98} 

Prescribed  by  ANSI  Std  Z39-18 


TABLE  OF  CONTENTS 


1.  INTRODUCTION . 6 

1.1  Background . 6 

1 .2  Objective  of  the  Demonstration . 6 

1.2.1  Objectives  of  the  ESTCP  UXO  Discrimination  Study . 6 

1.2.2  Technical  objectives  of  the  Discrimination  Study . 7 

1.3.2  Specific  Objective  of  the  Demonstration . 8 

2.  TECHNOLOGY  DESCRIPTION . 8 

2. 1  Technology  Development  and  Application . 8 

2.2  Previous  Testing  of  the  Technology . 16 

2.3  Advantages  and  Limitations  of  the  Technology . 16 

3.  DEMONSTARTION  DESIGN . 17 

3.1  Operational  Parameters  for  the  Technology . 17 

3.2  Period  of  Operation . 22 

4.  DATA  ANALYSIS  AND  INTERPRETATION . 24 

4. 1  UXO  Discrimination  Using  Training  Data . 30 

5.  PERFORMANCE  ASSESSMENT . 38 

6.  COST  ASSESSMENT . 43 

6.1  Cost  Analysis . 43 

7.  REFERENCES . 43 


2 


LIST  OF  FIGURES 


Figure  1 .  Berkeley  UXO  Discriminator  (BUD) . 9 

Figure  2.  Inversion  results  for  the  principal  polarizabilities,  location  and 

ORIENTATION  OF  81  MM  M821A1  PROJECTILE .  1 1 

Figure  3.  Inversion  results  for  the  principal  polarizabilities,  location  and 

ORIENTATION  OF  105  MM  M60  PROJECTILE .  1 1 

Figure  4.  Inversion  results  for  the  principal  polarizabilities,  location  and 

ORIENTATION  OF  19x8  CM  SCRAP  METAL .  12 

Figure  5 .  1 0%  uncertainty  in  location  as  a  function  of  object  diameter  and  depth  of 

THE  DETECTION  FOR  BUD  WITH  RECEIVERS  0.2  M  ABOVE  THE  GROUND .  13 

Figure  6.  1 0%  uncertainty  in  location  as  a  function  of  object  diameter  and  depth  of 

THE  DISCRIMINATION  FOR  BUD  WITH  RECEIVERS  0.2  M  ABOVE  THE  GROUND .  14 

Figure  7.  BUD  detection  plot  -  a  field  value  normalized  by  a  background  variation 
FOR  A  4.2”  MORTAR  AS  A  FUNCTION  OF  DEPTH  FOR  BUD  RECEIVERS  0.2  M  ABOVE  THE 
GROUND.  Solid  line  indicates  the  response  for  a  horizontal  orientation  of  the  4.2” 
MORTAR  (least  FAVORABLE).  DASHED  LINE  INDICATES  THE  RESPONSE  FOR  A  VERTICAL 

ORIENTATION  OF  THE  4.2”  MORTAR  (MOST  FAVORABLE) .  15 

Figure  8a.  Principal  polarizability  curves  as  a  function  of  time  for  Al  Sphere . 18 

Figure  8b.  Principal  polarizability  curves  as  a  function  of  time  for  Shotput  #1 . 18 

Figure  8c.  Principal  polarizability  curves  as  a  function  of  time  for  Shotput  #2 . 19 

Figure  8d.  Principal  polarizability  curves  as  a  function  of  time  for  Mortar  #1 . 19 

Figure  8e.  Principal  polarizability  curves  as  a  function  of  time  for  Mortar  #2 . 20 

Figure  9.  BUD  data  coverage  map  at  the  Former  Camp  Sibert,  AL.  Colored  areas  have 

CONTINUOUS  DATA  COVERAGE,  BLACK  PLUSES  INDICATE  LOCATIONS  OF  CUED 
MEASUREMENTS,  AND  BLUE  DOTS  REPRESENT  LOCATION  OF  TRAINING  DATA  SET  FOR  THE 

DISCRIMINATION . 22 

Figure  10.  BUD  GPO  detection  map . 26 

Figure  1 1 .  Photos  of  (a)  4.2”  mortar,  and  (b)  a  half-round . 26 

Figure  12.  Principal  polarizability  curves  as  a  function  of  time  for  4.2”  mortar . 27 

Figure  13.  Principal  polarizability  curves  as  a  function  of  time  for  a  half-round.  ...  27 

Figure  14.  BUD  detection  map  of  SEl  area . 28 

Figure  15.  Principal  polarizability  curves  as  a  function  of  time  for  a  small  scrap.  ...  29 

Figure  16.  Principal  polarizability  curves  as  a  function  of  time  for  a  base  plate . 29 

Figure  17;  ROC  curve  for  the  cued  targets  priority  dig  list . 39 

Figure  18:  ROC  curve  for  SEl  area  for  the  first  priority  dig  list . 40 

Eigure  19;  ROC  curve  for  SEl  area  for  the  second  priority  dig  list . 40 

Eigure  20:  Estimated  ROC  curve  for  cued  targets  priority  dig  list . 42 

Eigure  21 :  Estimated  ROC  curve  for  SEl  area  targets  priority  dig  list . 42 


3 


LIST  OF  TABLES 


TABLE  1:  Calibration  Targets . 17 

TABLE  2.  Total  time  of  major  demonstration  activities . 23 


4 


ACRONYMS 


AEM  Active  Electromagnetic  System 

BEID  Berkeley  EIXO  Discriminator 

DoD  Department  of  Defense 

ESTCP  Environmental  Security  Technology  Certification  Program 

EPGA  Eield  Programmable  Gate  Array 

PUDS  Pormerly  Used  Defense  Site 

GPO  Geophysical  Prove  Out 

GPS  Global  Positioning  System 

IDA  Institute  for  Defense  Analyses 

EBNE  Eawrence  Berkeley  National  Eaboratory 

RTK  Real  Time  Kinematic 

QA/QC  Quality  Assurance/Quality  Control 

SERDP  Strategic  Environmental  Research  and  Development  Program 

UXO  Unexploded  Ordnance 

YPG  Yuma  Proving  Ground 


5 


1.  INTRODUCTION 


1.1  Background 

The  FY06  Defense  Appropriation  eontains  funding  for  the  “Development  of  Advanced, 
Sophisticated,  Discrimination  Technologies  for  UXO  Cleanup”  in  the  Environmental  Security 
Technology  Certification  Program.  In  2003,  the  Defense  Science  Board  observed:  “The  ... 
problem  is  that  instruments  that  can  detect  the  buried  UXOs  also  detect  numerous  scrap  metal 
objects  and  other  artifacts,  which  leads  to  an  enormous  amount  of  expensive  digging.  Typically 
100  holes  may  be  dug  before  a  real  UXO  is  unearthed!  The  Task  Force  assessment  is  that  much 
of  this  wasteful  digging  can  be  eliminated  by  the  use  of  more  advanced  technology  instruments 
that  exploit  modern  digital  processing  and  advanced  multi-mode  sensors  to  achieve  an  improved 
level  of  discrimination  of  scrap  from  UXOs.” 

Significant  progress  has  been  made  in  discrimination  technology.  To  date,  testing  of  these 
approaches  has  been  primarily  limited  to  test  sites  with  only  limited  application  at  live  sites. 
Acceptance  of  discrimination  technologies  requires  demonstration  of  system  capabilities  at  real 
UXO  sites  under  real  world  conditions.  Any  attempt  to  declare  detected  anomalies  to  be 
harmless  and  requiring  no  further  investigation  require  demonstration  to  regulators  of  not  only 
individual  technologies,  but  of  an  entire  decision  making  process.  This  discrimination  study  was 
be  the  first  phase  in  what  is  expected  to  be  a  continuing  effort  that  will  span  several  years. 


1.2  Objective  of  the  Demonstration 

1,2.1  Objectives  of  the  ESTCP  UXO  Discrimination  Study 

As  outlined  in  the  Environmental  Security  Technology  Certification  Program  (ESTCP) 
Unexploded  Ordnance  (UXO)  Discrimination  Study  Demonstration  Plan,  the  objectives  of  the 
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study  were  twofold.  First,  the  study  was  designed  to  test  and  validate  UXO  detection  and 
discrimination  capabilities  of  currently  available  and  emerging  technologies  on  real  sites  under 
operational  conditions.  Second,  the  ESTCP  Program  Office  and  their  demonstrators  are 
investigating,  in  cooperation  with  regulators  and  program  managers,  how  UXO  discrimination 
technologies  can  be  implemented  in  cleanup  operations. 


1.2,2  Technical  objectives  of  the  Discrimination  Study 

The  study  was  designed  to  test  and  evaluate  the  capabilities  of  various  UXO  discrimination 
systems  each  of  which  consists  of  a  selected  sensor  hardware  system,  a  survey  mode,  and  a 
software-based  processing  step.  These  advanced  methods  will  be  compared  to  existing  practices 
and  will  validate  the  pilot  technologies  for  the  following: 

•  Detection  of  UXOs 

•  Identification  of  features  that  can  help  distinguish  scrap  and  other  clutter  from 
UXO 

•  Reduction  of  false  alarms  (items  that  could  be  safely  left  in  the  ground  that  are 
incorrectly  classified  as  UXO)  while  maintaining  acceptable  Pd’s 

•  Quantification  of  the  cost  and  time  impact  of  advanced  methods  on  the  overall 
cleanup  process  as  compared  to  existing  practices 

Additionally,  the  study  aims  to  understand  the  applicability  and  limitations  of  the  selected 
technologies  in  the  context  of  project  objectives,  site  characteristics,  and  suspected  ordnance 
contamination.  Sources  of  uncertainty  in  the  discrimination  process  will  be  identified  and  their 
impact  quantified  to  support  decision  making.  This  includes  issues  such  as  the  impact  of  data 
quality  due  to  how  the  data  are  collected.  The  process  for  making  the  dig  -  no  dig  decision 
process  will  be  explored.  Potential  quality  assurance/quality  control  (QA/QC)  processes  for 
discrimination  also  will  be  explored.  Finally,  high-quality,  well  documented  data  will  be 
collected  to  support  the  next  generation  of  signal  processing  research. 
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1,3,2  Specific  Objective  of  the  Demonstration 


The  demonstration  objeetive  was  to  determine  the  diserimination  eapabilities,  eost  and  reliability 
of  the  Berkeley  UXO  Diseriminator  (BUD).  Lawrenee  Berkeley  National  Laboratory  performed 
a  deteetion  and  diserimination  survey  of  the  SEl  area  (~  5  aeres)  of  the  Camp  Sibert  Formerly 
Used  Defense  Site  (FUDS)  in  Alabama.  In  addition,  BUD  was  used  in  a  eued  mode  to 
interrogate  200  seleeted  anomalies  within  Site  18  (SEl,  SE2,  and  SW  areas).  The  data  were 
eolleeted  in  aeeordanee  with  the  overall  study  demonstration  plan  ineluding  system 
characterization  with  the  emplaced  calibration  items  and  targets  in  the  Geophysical  Prove  Out 
(GPO). 


2.  TECHNOLOGY  DESCRIPTION 

2. 1  Technology  Development  and  Application 

The  Environmental  Security  Technology  Certification  Program,  ESTCP,  has  supported 
Eawrence  Berkeley  National  Eaboratory  (EBNL)  in  the  development  of  the  Berkeley  UXO 
Discriminator  (BUD)  that  not  only  detects  the  object  itself  but  also  quantitatively  determines  its 
size,  shape,  and  orientation.  Furthermore,  BUD  performs  target  characterization  from  a  single 
position  of  the  sensor  platform  above  a  target.  BUD  was  designed  to  detect  UXO  in  the  20  mm 
to  155  mm  size  range  for  depths  between  0  and  1.5  m,  and  to  characterize  them  in  a  depth  range 
from  0  to  1.1  m.  The  system  incorporates  three  orthogonal  transmitters,  and  eight  pairs  of 
differenced  receivers.  The  transmitter-receiver  assembly  together  with  the  acquisition  box,  as 
well  as  the  battery  power  and  global  positioning  system  (GPS)  receiver,  is  mounted  on  a  small 
cart  to  assure  system  mobility.  System  positioning  is  provided  by  state-of-the-art  Real  Time 
Kinematic  (RTK)  GPS  receiver.  The  survey  data  acquired  by  BUD  is  processed  by  software 
developed  by  EBNL,  which  is  efficient  and  simple,  and  can  be  operated  by  relatively  untrained 
personnel.  BUD  is  shown  in  Figure  1. 
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Figure  1,  Berkeley  UXO  Diseriminator  (BUD) 


Eight  receiver  coils  are  placed  horizontally  along  the  two  diagonals  of  the  upper  and  lower 
planes  of  the  two  horizontal  transmitter  loops.  These  receiver  coil  pairs  are  located  on  symmetry 
lines  through  the  center  of  the  system  and  each  pair  sees  identical  fields  during  the  on-time  of 
current  pulses  in  the  transmitter  coils.  They  are  wired  in  opposition  to  produce  zero  output 
during  the  on-time  of  the  pulses  in  three  orthogonal  transmitters.  Moreover,  this  configuration 
dramatically  reduces  noise  in  measurements  by  canceling  background  electromagnetic  fields 
(these  fields  are  uniform  over  the  scale  of  the  receiver  array  and  are  consequently  nulled  by  the 
differencing  operation),  and  by  canceling  noise  contributed  by  the  tilt  of  the  receivers  in  the 
Earth’s  magnetic  field,  and  greatly  enhances  receivers  sensitivity  to  gradients  of  the  target 
response. 

Data  acquisition  is  performed  on  a  single  board.  The  transmitter  coils  are  powered  by  circuits 
which  are  separate  from  the  data  acquisition  board.  These  pulsers  provide  resonant  circuit 
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switching  to  create  bi-polar  half-sine  pulses  of  350  ps  width.  The  current  peaks  at  18  A  whieh 
results  in  a  resonant  receiver  eireuit  voltage  of  '-'750  Volts.  The  operational  overall  half-sine 
duty  eycle  is  -'12%.  The  resonant  frequeney  of  the  inductive  load  is  -'90  kHz.  Transients  are 
digitized  with  a  sampling  interval  of  4  ps.  The  sensors  are  eritically  damped  6-ineh  325  turn 
loops  with  a  self-resonant  frequeney  of  25  kHz.  The  data  aequisition  board  has  12  high-speed 
ADC  ehannels.  Eight  of  these  ehannels  are  used  for  the  signal  from  receiver  eoils,  and  the 
remaining  four  ehannels  provide  information  about  the  system  (i.e.  tilt  information,  odometer). 

It  has  been  demonstrated  that  a  satisfactory  classification  scheme  is  one  that  determines  the 
prineipal  dipole  polarizabilities  of  a  target  -  a  near  intaet  UXO  displays  a  single  major 
polarizability  coineident  with  the  long  axis  of  the  objeet  and  two  equal  transverse  polarizabilities. 
The  indueed  moment  of  a  target  depends  on  the  strength  of  the  transmitted  indueing  field.  The 
moment  normalized  by  the  indueing  field  is  the  polarizability.  This  deseription  of  the  inherent 
polarizabilities  of  a  target  constitutes  a  major  advanee  in  diseriminating  UXO  from  irregular 
scrap  metal.  Figures  2-4  illustrate  a  diserimination  eapability  of  the  system  for  UXO  objects 
(Figures  2  and  3),  and  scrap  metal  (Figure  4).  All  three  figures  have  estimated  prineipal 
polarizabilities  as  a  function  of  time  plotted  on  the  left,  values  of  true  and  estimated  location  and 
orientation  on  the  right,  and  object  images  at  the  bottom.  While  UXO  objeets  have  a  single 
major  polarizability  eoincident  with  the  long  axis  of  the  object  and  two  equal  transverse 
polarizabilities  (Figure  2-3),  the  scrap  metal  exhibits  three  distinct  principal  polarizabilities 
(Figure  4).  The  loeations  and  orientations  are  reeovered  within  a  few  percent  of  true  values  for 
all  three  objects. 

These  results  elearly  show  that  a  multiple  transmitter  -  multiple  reeeiver  system  ean  resolve  the 
intrinsie  polarizabilities  of  a  target  and  that  there  are  very  clear  distinetions  between  symmetric 
intact  UXO  and  irregular  scrap  metal. 
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Figure  2,  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  81  mm 
M821A1  projectile 


Figure  3,  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  105  mm 
M60  projectile 
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Figure  4,  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  19x8  cm 
scrap  metal 


The  detection  performance  of  the  system  is  governed  by  a  size-depth  curve  shown  in  Figure  5. 
This  curve  was  calculated  for  BUD  assuming  that  the  receiver  plane  is  0.2  m  above  the  ground. 
Figure  5  shows  that,  for  example,  BUD  can  detect  an  object  with  0.1  m  diameter  down  to  the 
depth  of  0.9  m  with  depth  uncertainty  of  10%.  Any  objects  buried  at  a  depth  of  more  than  1.3  m 
will  have  a  low  probability  of  detection.  The  discrimination  performance  of  the  system  is 
governed  by  a  size-depth  curve  shown  in  Figure  6.  Again,  this  curve  was  calculated  for  BUD 
assuming  that  the  receiver  plane  is  0.2  m  above  the  ground.  Figure  6  shows  that,  for  example, 
BUD  can  discriminate  an  object  with  0.1  m  diameter  down  to  the  depth  of  0.63  m  with  depth 
uncertainty  of  10%.  Any  objects  buried  at  the  depth  more  than  0.9  m  will  have  a  low  probability 
of  discrimination. 

Object  orientation  estimates  and  equivalent  dipole  polarizability  estimates  used  for  large  and 
shallow  UXO/scrap  discrimination  are  more  problematic  as  they  are  affected  by  higher  order 
(non-dipole)  terms  induced  in  objects  due  to  source  field  gradients  along  the  length  of  the 
objects.  For  example,  a  vertical  0.4  m  object  directly  below  the  system  needs  to  be  about  0.90  m 
deep  for  perturbations  due  to  gradients  along  the  length  of  the  object  to  be  of  the  order  of  20  % 
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of  the  uniform  field  objeet  response.  Similarly,  vertieal  objects  0.5  m,  and  0.6  m  long  need  to  be 
1.15  m,  and  1.42  m,  respectively,  below  the  system.  For  horizontal  objects  the  effect  of 
gradients  across  the  objects'  diameter  are  much  smaller.  For  example,  155  mm  and  105  mm 
projectiles  need  to  be  only  0.30  m,  and  0.19  m,  respectively,  below  the  system.  A  polarizability 
index  (in  cm  ),  which  is  an  average  value  of  the  product  of  time  (in  seconds)  and  polarizability 
rate  (in  mVs)  over  the  35  sample  times  logarithmically  spaced  from  153  to  1387  ps,  and  three 
polarizabilities,  can  be  calculated  for  any  object.  We  used  this  polarizability  index  to  decide 
when  the  object  is  in  a  uniform  source  field.  Objects  with  the  polarizability  index  smaller  than 
600  cm  and  deeper  than  1.8  m  below  BUD,  or  smaller  than  200  cm  and  deeper  than  1.35  m,  or 
smaller  than  80  cm  and  deeper  than  0.90  m,  or  smaller  than  9  cm  and  deeper  than  0.20  m  below 
BUD  are  sufficiently  deep  that  the  effects  of  vertical  source  field  gradients  should  be  less  than 
15%.  All  other  objects  are  considered  large  and  shallow  objects. 


Depth  of  detection  for  1:3.5  aspect  ratio  steel  sheils  (spheroid  model) 


Figure  5.  10%  uncertainty  in  location  as  a  function  of  object  diameter  and  depth  of  the  detection 
for  BUD  with  receivers  0.2  m  above  the  ground 
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Depth  of  polarizability  resolution  to  10%  for  horizontal  1:3.5  aspect  ratio  steel  shells 


Figure  6,  10%  uncertainty  in  location  as  a  function  of  object  diameter  and  depth  of  the 
discrimination  for  BUD  with  receivers  0.2  m  above  the  ground 


To  assure  proper  object  identification  and  UXO/scrap  discrimination,  in  the  case  of  large  and 
shallow  objects,  we  took  measurements  at  five  sites  spaced  0.5  m  along  a  line  traversing  the 
object.  Initially,  object  orientation  was  estimated  from  the  response  at  the  most  distant  of  these 
sites.  Then,  the  site,  for  which  the  line  from  the  object  center  to  the  BUD  bottom  receiver  plane 
center  that  was  closest  to  being  90°  to  the  orientation  of  the  objects'  interpreted  axis  of  greatest 
polarizability,  was  selected.  The  data  from  this  site  have  the  smallest  source  field  gradients  in 
the  direction  of  the  estimated  axis  of  greatest  polarizability.  The  results  of  polarizability 
inversion  from  this  site  was  used  for  object  classification. 

At  Camp  Sibert  the  primary  UXO  targets  expected  were  4.2”  (107  mm)  mortars  that  are  about 
0.4  m  long.  The  BUD  detection  threshold  is  based  on  the  signal  strength  relative  to  levels  of 
background  response  variation  observed  at  Yuma  Proving  Ground.  Measured  signal  strengths 
(field  value)  normalized  by  this  background  variation  for  a  4.2”  mortar  as  a  function  of  depth  are 
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shown  in  Figure  7.  The  solid  line  indicates  the  response  of  a  4.2”  mortar  in  a  horizontal  (least 
favorable)  orientation,  and  the  dashed  line  indicated  the  response  of  a  4.2”  mortar  in  a  vertical 
(most  favorable)  orientation.  The  detection  threshold  was  set  to  10,  which  is  50%  of  the  value 
that  would  be  measured  for  the  4.2”  mortar  at  the  depth  equal  to  1 1  x  diameter  of  the  mortar. 


Figure  7.  BUD  detection  plot  -  a  field  value  normalized  by  a  background  variation  for  a  4.2” 
mortar  as  a  function  of  depth  for  BUD  receivers  0.2  m  above  the  ground.  Solid  line  indicates  the 
response  for  a  horizontal  orientation  of  the  4.2”  mortar  (least  favorable).  Dashed  line  indicates 
the  response  for  a  vertical  orientation  of  the  4.2”  mortar  (most  favorable). 


Since  scrap  from  exploded  4.2"  mortars  is  in  general  significantly  smaller  than  these,  and  since 
smaller  objects  of  similar  composition  generally  have  smaller  polarizability  responses,  a  simple 
criterion  for  scrap/UXO  in  this  case  is  based  on  the  magnitude  of  polarizability  responses.  For 
the  4.2”  mortar  the  polarizability  index  is  larger  than  300  cm  .  Consequently,  any  polarizability 
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responses  from  Camp  Sibert  with  a  polarizability  index  greater  than  150  cm  were  considered 
conservatively  as  most  likely  due  to  a  UXO  or  a  large  fragment  thereof.  In  addition  to  this, 
polarizability  responses  were  matched  to  catalog  responses  as  described  in  Chapter  4. 


2.2  Previous  Testing  of  the  Technoiogy 

The  performance  of  the  BUD  has  been  demonstrated  at  a  local  test  site  in  California,  as  well  as  at 
the  Calibration  and  Blind  Test  Grids  and  the  Open  Field  Range  at  the  Yuma  Proving  Ground 
(YPG),  Arizona.  The  results  have  been  presented  at  various  meetings  and  published  in  scientific 
journals. 


2.3  Advantages  and  Limitations  of  the  Technoiogy 

This  is  the  first  AEM  system  that  can  not  only  detect  UXO  but  also  discriminate  it  from  non- 
UXO/scrap  and  give  its  characteristics  (location,  size,  polarizability).  Moreover,  the  object  can 
be  characterized  from  a  single  position  of  the  sensor  platform  above  the  object.  BUD  was 
designed  to  detect  UXO  in  the  20  mm  to  155  mm  size  range  buried  anywhere  from  the  surface 
down  to  1.5  m  depth.  Any  objects  buried  at  the  depth  more  than  1.5  m  will  have  a  low 
probability  of  detection.  In  addition,  BUD  was  designed  to  characterize  UXO  in  the  same  size 
range  in  depths  between  0  and  I.I  m.  Any  objects  buried  at  the  depth  more  than  I.I  m  will  have 
a  low  probability  of  discrimination.  With  existing  algorithms  in  the  system  computer  it  is  not 
possible  to  recover  the  principal  polarizabilities  of  large  objects  close  to  the  system.  Detection  of 
large  shallow  objects  is  assured,  but  at  present  discrimination  is  not.  Post  processing  of  the  field 
data  is  required  for  shape  discrimination  of  large  shallow  targets.  See  Chapter  2.1  for  details. 
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3.  DEMONSTARTION  DESIGN 


3.1  Operational  Parameters  for  the  Technology 


We  shipped  all  the  equipment  and  supplies  to  the  test  site  using  a  eontainer  and  a  eommercial 
trueking  company.  Personnel  flew  and  drove  to  the  site  in  rented  vehicles.  Equipment  was 
stored  in  a  support  building  provided  by  the  host  facility.  The  demonstration  team  consisted  of  2 
people,  and  a  PI  was  there  at  the  beginning  of  the  survey. 


Assembling  the  cart,  connecting  the  batteries,  checking  the  data  acquisition  system  and  verifying 
the  data  records  took  about  30  minutes  every  morning.  This  was  followed  by  about  30  minutes 
system  calibration  along  the  calibration  line  established  be  the  ESTCP  office.  This  line  was 
measured  every  morning  and  evening.  Responses  of  all  calibration  targets  were  consistent  and 
repeatable  throughout  the  survey.  The  list  of  calibration  targets  is  given  in  Table  1. 

TABLE  1:  CALIBRATION  TARGETS 


Item 

Easting 

(m) 

Northing 

(m) 

Depth 

(cm) 

Grid 

Orientation 

(deg) 

Length 

(m) 

4"  Al  Sphere 

578849.204 

3752001.662 

6 

N/A 

N/A 

Shotput  #1 

578846.064 

3751994.346 

10 

N/A 

N/A 

Shotput  #2 

578843.346 

3751989.040 

20 

N/A 

N/A 

4.2"  Mortar  #1 

578839.851 

3751981.456 

35 

293 

0.469 

4.2"  Mortar  #2 

578836.607 

3751975.109 

57 

302 

0.394 

The  principal  polarizability  curves  as  a  function  of  time  for  targets  listed  in  Table  1  are  shown  in 
Eigures  8a-8e. 
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Figure  8a,  Principal  polarizability  curves  as  a  function  of  time  for  A1  Sphere. 


Figure  8b,  Principal  polarizability  curves  as  a  funetion  of  time  for  Shotput  #1. 
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Principal  Polarizabilities  (m''3/s) 


Figure  8c,  Principal  polarizability  curves  as  a  function  of  time  for  Shotput  #2. 


Figure  8d,  Principal  polarizability  curves  as  a  function  of  time  for  Mortar  #1. 


19 


Figure  8e,  Principal  polarizability  curves  as  a  function  of  time  for  Mortar  #2. 

We  demonstrated  two  modes  of  BUD  operation  -  (1)  simultaneous  detection  and 
characterization/  discrimination,  and  (2)  the  cued  mode.  Mode  (1)  was  used  over  a  calibration 
strip,  the  GPO,  and  5  acres  of  SEl  area.  The  survey  area  was  divided  into  a  hundred  ~200-m 
long  lines  in  a  more-less  east-west  direction.  Line  spacing  in  the  orthogonal  direction  (north- 
south)  was  1  m.  We  conducted  the  survey  at  a  speed  of  0.5  m/s.  Five  200  m  long  string  lines 
were  laid  out  at  any  point  in  time  to  help  the  operator  with  BUD  navigation  and  to  assure  full 
area  coverage.  BUD  was  pushed  along  the  line  at  a  constant  speed  in  a  search  mode.  Data  were 
recorded  and  stored  continuously.  In  principle,  any  object  within  the  1  m  x  1  m  footprint  of  the 
horizontal  transmitter  coil  and  1.2  m  in  front  of  the  system  can  be  detected  and  characterized. 
The  detection  threshold  was  set  to  10,  and  it  was  based  on  the  signal  strength  relative  to  levels  of 
background  response  variation  observed  at  Yuma  Proving  Ground  and  the  GPO  area  at  Camp 
Sibert.  In  the  search  mode  the  operator  was  alerted  to  the  presence  of  a  target  every  time  the 
signal  level  exceeded  the  detection  threshold.  The  threshold  value  was  recorded  together  with 
data  file  name,  and  acquisition  time,  and  BUD  GPS  location.  If  the  target  was  inside  of  the  BUD 
footprint,  the  operator  stopped  and  a  full  sequence  of  measurements  was  initiated.  The  three 
discriminating  polarizability  responses  were  recorded  and  visually  presented  on  the  computer 
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screen.  The  depth  and  horizontal  loeation  with  respect  to  the  eart  were  reeorded,  together  with 
the  GPS  loeation  of  the  referenee  point  on  the  eart.  Then  the  eart  again  moved  at  a  eonstant 
speed  in  seareh  mode  until  the  next  target  was  detected  and  the  diserimination  proeess  was 
repeated.  This  mode  of  operation  has  the  advantage  that  target  reaequisition  is  not  neeessary  for 
eharaeterization.  As  deseribed  in  Chapter  2.1,  objeet  orientation  estimates  and  equivalent  dipole 
polarizability  estimates  used  for  large  and  shallow  UXO/serap  diserimination  are  more 
problematie  as  they  are  affeeted  by  higher  order  (non-dipole)  terms  indueed  in  objeets  due  to 
souree  field  gradients  along  the  length  of  the  objeets.  In  the  ease  when  a  large  shallow  objeet 
was  found,  we  eolleeted  five  measurements  spaeed  0.5  m  along  a  line  traversing  the  objeet  (i.e.  if 
the  objeet  loeation  was  at  0.0,  measurements  were  taken  at  1.0  m,  0.5  m,  0.0  m,  -0.5  m,  and  -1.0 
m)  so  that  system  got  further  away  from  the  object,  and  hence  minimized  souree  gradients,  at  one 
or  more  loeations.  The  measurement  that  best  satisfied  the  eriteria  deseribed  earlier  was  used  for 
the  objeet  eharaeterization. 

In  addition,  we  performed  a  eued  survey  over  200  objeets  seleeted  by  the  ESTCP  Offiee.  In  this 
ease  BUD  was  brought  to  marked  loeations  and  ran  in  the  diserimination  mode.  The  three 
diseriminating  polarizability  responses  were  reeorded  and  visually  presented  on  the  eomputer 
sereen.  The  depth  and  horizontal  loeation  with  respeet  to  the  eart  was  reeorded,  together  with  a 
GPS  loeation  of  the  referenee  point  on  the  eart.  As  described  earlier,  for  large  shallow  objeets 
we  eolleeted  five  measurements  spaeed  0.5  m  along  a  line  traversing  the  objeet  (i.e.  if  the  object 
location  was  at  0.0,  measurements  were  taken  at  1.0  m,  0.5  m,  0.0  m,  -0.5  m,  and  -1.0  m).  The 
measurement  that  best  satisfied  the  eriteria  deseribed  earlier  was  used  for  the  objeet 
eharaeterization. 

Figure  9  shows  areas  eovered  in  this  survey,  as  well  as  loeation  of  training  data  for  the 
diserimination.  Colored  areas  have  eontinuous  data  eoverage,  blaek  pluses  indicate  loeations  of 
cued  measurements,  and  blue  dots  represent  loeation  of  training  data  set  for  the  diserimination. 
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Figure  9,  BUD  data  coverage  map  at  the  Former  Camp  Sibert,  AL.  Colored  areas  have 
eontinuous  data  eoverage,  blaek  pluses  indieate  loeations  of  eued  measurements,  and  blue  dots 
represent  loeation  of  training  data  set  for  the  diserimination. 

3.2  Period  of  Operation 

The  survey  was  performed  between  May  1,  2007  and  May  22,  2007.  The  time  required  for  the 
major  activities  involved  in  the  field  demonstration  are  provided  in  Table  2. 

The  following  GPS  eoordinates  were  used  for  BUD  positioning.  For  the  GPO  area,  the  north 
part  of  the  SEl  area,  and  SW  area  our  base  station  was  loeated  at  (Easting,  Northing)  =  (578833, 
3751629).  For  the  south  part  of  the  SEl  area,  and  SE2  area  the  base  station  was  loeated  at 
(Easting,  Northing)  =  (578807,  3751707).  These  GPS  locations  were  provided  to  us  by  the 
ESTCP  office. 
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Demobilization  consisted  of  disassembly  and  removal  of  the  system  which  took  about  2  hours. 
BUD  and  all  materials  and  supplies  were  removed  from  the  site  and  shipped  in  the  container 
back  to  Berkeley,  California  after  the  survey. 


TABLE  2.  TOTAL  TIME  OF  MAJOR  DEMONSTRATION  ACTIVITIES 


Task 

Time 

Notes 

Site  Orientation,  Safety  Briefing 

3  hr 

GPS  Base  Setup 

8.5  hr 

30  min/day 

Field  Checks  &  Calibrations 

17  hr 

1  hr/day  system  and  background 

calibration 

GPO 

3  days 

1  acre 

Detection  and  Discrimination 

Survey 

9  days 

5  acres 

Cued  Survey 

3  days 

200  targets 

Data  Processing,  Quality 

Assurance  &Archiving 

1 8  days 

Contingencies 

3  days 

background  response  variations, 

repeat  measurements,  weather 

related  issues 

Total  Time 

1 8  days 
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4.  DATA  ANALYSIS  AND  INTERPRETATION 


The  first  step  prior  to  data  eolleetion  was  a  system  calibration  and  a  background  level  estimation. 
We  measured  the  background  noise  with  the  transmitters  in  the  off-position  and  calculated  the 
spectrum  for  all  channels.  Then  we  turned  the  transmitters  on  and  measured  the  background 
field  on  all  channels.  This  step  was  repeated  at  least  twice  to  make  sure  the  background  field 
was  stable  and  could  be  used  as  the  baseline  measurement  that  was  subtracted  from  the  data. 
The  next  step  was  to  take  data  over  calibration  targets  on  the  calibration  line.  While  surveying 
the  GPO  area  we  encountered  significant  variations  in  the  background,  possible  due  to  presence 
of  magnetic  soil,  and  we  had  to  modify  our  algorithm  to  account  for  that.  The  SEl  area  didn’t 
suffer  from  these  background  variations. 

Twelve  channels  of  field  data  are  recorded  at  a  rate  of  250  k-samples/second  for  each  of  three 
transmitters.  Field  data  were  stacked  together  in  a  field  programmable  gate  array  (FPGA)  and 
transferred  to  a  field  computer  (laptop)  forming  a  primitive  stack,  labeled  with  header 
information  (instrument  position,  tilt  and  heading,  odometer,  time  stamp,  channels  of  transmitter 
current,  etc).  An  even  number  of  primitive  stacks  was  averaged  together  to  form  stacked  data  for 
further  processing. 

The  peak  transmitter  current  was  estimated  from  the  stacked  transmitter  current  channel  record, 
and  the  data  were  normalized  by  that  value.  Nominal  transmitter  shut-off  time  was  estimated, 
and  induction  responses  were  computed  at  35  logarithmically  spaced  times  between  153  and 
1283  ps,  averaged  in  half-sine  windows  with  widths  10%  of  the  center  time  after  transmitter 
pulse  shut-off  Responses  were  differenced  with  background  responses  collected  over  a  nearby 
site  determined  to  be  relatively  free  of  metallic  objects  by  having  a  system  response  which  varies 
little  with  system  translation.  Error  bars  were  computed  for  these  based  on  the  scatter  in  raw  4 
ps  samples  at  late  time,  and  used  to  estimate  uncertainties  in  field  inversions  of  equivalent  dipole 
polarizability  responses  and  object  position.  These  uncertainties  are  overly  optimistic,  as  a 
greater  source  of  noise  is  a  variation  in  the  system  background  response. 
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In  post  processing,  the  data  were  reproeessed,  with  the  baekground  response  for  eaeh  line  of  data 
computed  using  a  trimmed  median  response  at  each  time  and  reeeiver,  for  eaeh  transmitter 
response.  The  trimmed  median  used  is  the  median  of  all  points  within  2  median  absolute 
deviations  (MAD)  of  the  (untrimmed)  median  response  at  eaeh  time,  reeeiver,  and  transmitter. 
As  the  baekground  variation  is  a  larger  souree  of  uncertainty  than  the  previously  mentioned  noise 
estimated  from  seatter  in  raw  4  |us,  1 .48  times  the  MAD  in  individual  line  responses  was  used  as 
the  estimated  noise  level  in  later  proeessing.  The  resulting  24  ehannels  of  normalized  responses 
were  then  inverted  for  eandidate  objeet  position  and  prineipal  polarizabilities  as  a  funetion  of 
time  after  transmitter  shut-off  Data  before  140  |us  were  ignored. 

The  first  area  we  surveyed  was  the  GPO  area.  The  detection  map  is  shown  in  Figure  10.  The 
color  represents  strength  of  the  measured  response  normalized  by  the  baekground  variation 
(same  property  as  the  y-axis  value  in  Figure  7).  The  deteetion  threshold  was  set  to  10,  therefore 
red  and  yellow  colors  represent  the  baekground  response,  while  green  and  blue  eolors  indieate 
loeations  of  metallie  objeets.  Pluses  indieate  loeations  of  4.2”  mortars,  while  eireles  indieate 
loeations  of  half-rounds.  The  GPO  area  was  seeded  with  these  items  in  order  to  test  eapabilities 
and  limitations  of  seleeted  teehnologies  ineluded  in  this  demonstration.  Henee,  although  BUD 
deteeted  additional  metallie  objeets  in  the  area  they  were  not  exeavated  and  we  have  no  ground 
truth  for  them.  A  picture  of  a  typical  4.2”  mortar  is  shown  in  Figure  11a,  while  a  picture  of  a 
half-round  is  shown  in  Figure  1  lb.  Prineipal  polarizability  eurves  as  a  function  of  time  for  both 
objects  are  shown  in  Figure  12  and  Figure  13,  respeetively.  Figures  12  and  13  illustrate  the 
diserimination  eapability  of  prineipal  polarizabilities  between  symmetric  UXO,  in  this  case  the 
4.2”  mortar,  and  scrap,  in  this  case  the  half-round.  The  4.2”  mortar  has  a  single  major 
polarizability  eoineident  with  the  long  axis  of  the  objeet  and  two  equal  transverse  polarizabilities 
(Figure  12);  the  half-round  has  three  distinet  prineipal  polarizabilities  (Figure  13).  As  mentioned 
above  this  area  had  a  highly  variable  baekground,  espeeially  at  its  north  edge.  This  had  an  effeet 
on  a  signal  level  from  deep  4.2”  mortars,  and  in  turn  on  our  inversion  results  whieh  made 
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identification  of  these  mortars  challenging.  None  of  the  anomalies  that  were  not  dug  had 
response  either  of  the  4.2”  mortar  or  the  half-round;  they  were  all  considered  scrap. 
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Figure  10,  BUD  GPO  detection  map. 


(a)  (b) 

Figure  11,  Photos  of  (a)  4.2”  mortar,  and  (b)  a  half-round. 
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Figure  12,  Principal  polarizability  curves  as  a  function  of  time  for  4.2”  mortar. 


Figure  13,  Prineipal  polarizability  eurves  as  a  funetion  of  time  for  a  half-round. 

The  SEl  area  was  surveyed  in  our  preferred  mode  of  operation  -  detection  and  discrimination  at 
the  same  time  (see  Chapter  3.1  for  a  description  of  our  survey  procedure).  Figure  14  shows  the 
detection  map  of  this  area.  Again,  the  deteetion  threshold  was  set  to  10,  therefore  red  and  yellow 
eolors  represent  the  background  response,  while  green  and  blue  eolors  indicate  locations  of 
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Figure  14,  BUD  detection  map  of  SEl  area. 
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metallic  objects.  The  detection  list  contained  every  object  with  the  response  above  the  detection 
threshold  or  polarizability  index  larger  than  150  cm  .  We  identified  358  such  locations. 


In  addition  to  the  responses  of  4.2”  mortars  and  half-rounds  shown  in  Figure  12  and  13,  the  area 
contained  a  lot  of  scrap  metal  and  some  representative  responses  are  shown  in  Figure  15  and  16. 
All  the  objects  have  distinct  polarizability  signatures  which  allows  for  a  clear  discrimination 
between  4.2”  mortars  and  scrap. 


Figure  15,  Principal  polarizability  curves  as  a  function  of  time  for  a  small  scrap. 


Figure  16,  Principal  polarizability  curves  as  a  function  of  time  for  a  base  plate. 
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For  the  discrimination  of  cued  targets  and  objects  from  the  SEl  area  ESTCP  office  provided  us 
with  a  training  data  set.  This  data  set  contained  100  objects  total  -  30  mortars  and  8  half-rounds 
from  the  GPO  area,  and  additional  15  mortars,  5  half-rounds,  and  42  scrap  metals  from  the  SEl, 
SE2,  and  SW  areas.  The  location  of  the  training  data  is  shown  in  blue  in  Eigure  9.  A  detailed 
description  of  the  discrimination  process  is  given  in  following  section  (Chapter  4.1). 

Erom  200  cued  survey  locations  27  objects  were  in  the  training  data  set,  and  23  of  them  were 
identified  as  multiple  objects  and  therefore  excluded  from  the  set.  We  provided  discrimination 
results  for  remaining  150  objects  in  form  of  two  priority  dig  lists.  In  the  first  one  the  ‘stop 
digging’  point  was  when  any  object  past  this  mark  had  a  very  little  probability  of  being  4.2” 
mortar.  This  list  indicated  that  90  objects  can  be  left  in  the  ground,  and  60  objects  have  to  be 
dug.  The  second  priority  dig  list  had  the  ‘stop  digging’  point  at  90%  probability  of  any  object 
being  4.2”  mortar.  In  this  list  only  43  objects  were  identified  as  ‘need  to  dig’  while  107  could  be 
left  safely  in  the  ground. 

The  SEl  area  discrimination  set  contained  266  objects.  Again,  we  produced  two  priority  dig 
lists.  The  first  one  with  the  ‘stop  digging’  point  when  any  object  past  this  mark  had  very  little 
probability  of  being  4.2”  mortar,  and  the  second  one  with  the  ‘stop  digging’  point  at  90% 
probability  of  any  object  being  4.2”  mortar.  In  the  first  priority  dig  list  we  indicated  that  75 
objects  have  to  be  dug,  while  191  objects  can  be  left  in  the  ground.  The  second  priority  list 
classified  63  objects  as  ‘need  to  dig’  and  203  objects  as  scrap. 


4.1  UXO  Discrimination  Using  Training  Data 

The  release  of  object  identities  for  a  discrimination  training  data  set  allowed  for  exploration  of  a 
more  extensive  feature  set  than  the  simple  averaged  product  of  polarizability  and  time  initially 
proposed.  Training  data  consisted  of  a  set  of  principal  polarizability  responses  at  34 
logarithmically  spaced  times  centered  from  153  ps  to  1300  ps  after  transmitter  shut-off,  from 
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objects  identified  as  UXO  (mortars),  large  fragments  of  UXO  (half-rounds,  half-shells),  scrap 
(clutter),  and  cultural  items  (such  as  wrenches).  Discrimination  was  made  for  two  separate 
problems;  discriminating  mortar  and  half-round  responses  from  scrap  responses,  and 
discriminating  mortar  responses  from  half-round  and  scrap  responses.  We  treated  the 
mathematics  of  these  two  problems  identically,  and  both  were  treated  below  as  the  general 
problem  of  discriminating  ‘mortar’  responses  from  ‘scrap’  responses.  Since  there  was  an 
insufficient  number  of  responses  of  cultural  items  found  in  area,  we  were  not  able  to  develop  a 
reasonable  estimate  of  the  distribution  for  their  responses,  so  they  were  ignored;  any  cultural 
item  responses  to  be  classified  later  will  be  classified  ‘arbitrarily’  as  ‘scrap’  or  ‘mortars’. 

The  data  time  interval  was  sub-divided  logarithmically  into  a  number  Udiv  of  sub-intervals  (e.g., 
6).  The  product  of  each  principal  polarizability  with  its  sample  time  was  averaged  over  each  of 
these  intervals.  Since  there  are  three  principal  polarizabilities,  this  results  in  nfeat  =  3ndiv  reduced 
data,  hence  forth  called  ‘features’.  The  number  of  sub-divisions  naiv  was  chosen  using  cross 
validation.  In  cross  validation,  an  analysis  method  is  applied  to  most  of  a  training  data  set  and 
the  results  are  used  to  predict  something  about  the  remaining  (excluded)  training  data.  This  is 
done  many  times,  excluding  a  different  set  of  training  data  each  time,  and  a  choice  made,  e.g., 
the  value  of  Udiv,  based  on  what  gives  the  best  predictions  averaged  over  many  times. 

In  our  application,  two  thirds  of  the  training  data  were  randomly  selected  for  direct  use  in 
training,  and  one  third  was  reserved  for  later  calibration  and  is  termed  ‘reserved  data’.  Based  on 
the  selected  2/3  of  the  training  data,  the  probability  that  an  observation  is  due  to  ‘scrap’  is 
estimated  from  a  ratio  of  empirical  probability  density  estimates  for  feature  values.  Empirical 
probability  distributions  are  probability  distribution  estimates  made  based  on  observed  data. 
Empirical  probability  distributions  will  be  constructed  below. 

Within  the  selected  2/3  of  the  training  data  (‘non-reserved’  data),  additional  responses  were 
randomly  selected  for  exclusion  in  cross  validation,  and  are  termed  ‘excluded  data’.  In  this 
work,  a  constant  number  of  ‘mortar’  training  data,  and  a  constant  number  of  ‘scrap’  training  data 
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were  excluded  at  a  time,  in  roughly  equal  proportions.  The  number  of  excluded  data  was  chosen 
so  that  one  response  was  withheld  at  a  time  from  the  smaller  of  the  sets  of  non-reserved  ‘mortar’ 
and  ‘scrap’  training  data.  For  example,  in  mortars  vs.  scrap  (this  includes  half-rounds) 
discrimination  problem,  we  used  45  mortar  training  responses  and  55  scrap  responses.  Two 
thirds  were  used  for  training  -  30  mortars  and  37  scrap  responses;  one  third  -  15  mortars  and  18 
scrap  responses  were  not  used  in  training,  but  were  reserved  for  later  use.  In  cross  validation  one 
mortar  and  one  scrap  were  withheld  from  the  training  responses  at  a  time.  This  was  cycled 
through  30  choices  of  excluded  responses  excluding  each  mortar  and  most  scrap  responses  once. 

We  estimate  empirical  probability  densities  separately  from  ‘mortar’  and  ‘scrap’  classes.  In 
constructing  an  empirical  probability  distribution  for  either  ‘mortar’  or  ‘scrap’  responses,  it  is 
desirable  to  smear  probability  associated  with  any  particular  data  point  into  a  region  centered 
around  it,  as  it  is  extremely  unlikely  that  another  data  point  will  have  a  value  somewhere  in  the 
neighborhood.  To  get  an  idea  of  how  much  to  smear  out  the  probability  associated  with  each 
data  point  in  forming  an  empirical  probability  distribution,  a  trimmed  cross  power  matrix  is 
constructed  for  variation  of  data  vectors  about  their  median  values.  To  do  this,  within  training 
data  for  a  class,  values  of  each  feature  are  sorted,  the  median  and  median  absolute  deviation 
(MAD)  from  the  median  are  noted.  In  finding  these  medians  and  MADs,  responses  reserved  for 
calibration  and  responses  excluded  for  cross-validation  are  omitted.  The  values  for  each  feature 
are  then  shifted  by  subtracting  its  median  value.  As  the  scale  of  different  features  may  vary  from 
feature  to  feature,  before  forming  the  trimmed  cross  power  matrix,  the  shifted  feature  values  for 
each  feature  are  normalized  by  dividing  their  MAD  resulting  in  shifted  normalized  feature 
vectors,  for  mortar  responses,  and  for  scrap  responses.  Trimmed  cross  power 

matrices  are  computed  from  these,  with  superscript  meaning  either  or  To 

compute  the  trimmed  cross  power  matrix  for  a  class,  values  of  are  computed  for  the 

class’s  training  data  and  sorted  to  find  median(|vi'^'^‘^”*’'|),  omitting  reserved  and  excluded  data. 
Then  the  class  trimmed  cross  power  matrices  are  computed  as 
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where  and  n^‘^'^’^®^rsrv,  is  the  number  of  (elass)  responses 

reserved  for  subsequent  ealibration,  and  is  the  number  of  (elass)  responses  exeluded  as 

a  part  of  eross  validation.  The  sums  are  over  all  non-exeluded  non-reserved  class  responses,  t 
denotes  transpose,  and  median  means  median(|vi*^*‘^'^“'’^|).  In  the  second  sum,  the  contribution  of 
large  magnitude  feature  vectors  are  downweighted.  Feature  vector  probability  density 

function  is  estimated  empirically  as  proportional  to 


p(class)^.y(class)^  _  jA  ^ 


iin  class 
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(2) 


with 


JA  _  ^~(class)yi+nfj„)/nf„, 

where  superscript  has  been  omitted  from  and  in  the  denominator,  and  reserved 

and  excluded  terms  omitted  from  the  sum.  The  quantity  in  Eq.  (2)  will  be  referred  to  as  a 
density,  although  strictly  speaking  a  density  is  normalized  to  have  unit  integral,  and  quantity  in 
Eq.  (2)  has  not  been  normalized.  Eq.  (2)  is  a  generalization  of  a  Cauchy  distribution,  with  the 
outer  exponent  being  the  smallest  half  integer  value  yielding  a  finite  variance. 


In  cross  validation,  densities  (Eq.  (2))  are  computed  for  ‘mortar’  and  ‘scrap’  classes  from  non- 
exeluded  responses,  and  feature  vectors  and  are  computed  for  excluded  training 

responses  not  reserved  for  calibration,  where  for  the  j’th  response,  the  two  differ  in  component 
offsets  and  normalizations.  The  first  is  used  in  estimating  the  response’s  likelihood  as  a  ‘mortar’ 
response,  and  the  second  in  estimating  its  likelihood  as  a  ‘scrap’  response.  Eor  a  given  response, 
assuming  that  the  proportionality  constant  is  the  same  for  both  ‘mortar’  and  ‘scrap’  estimated 
densities,  the  probability  that  the  response  is  due  to  a  ‘scrap’  would  be 


p  (scrap)  (scrap)  ^ 


'(mort)^(mort)  p(scrap)  ^.^0 


(scrap) 


(4) 
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as  the  common  proportionality  factor  cancels  in  their  ratio.  The  probability  that  is  due  to 
‘mortar’,  would  be  one  less  this  number.  Allowing  for  a  ratio  of  proportionality  constants  to  be 
a^,  the  densities  are  then  and  (within  a  common  scale  factor), 

and  the  probability  that  the  response  is  due  to  ’scrap’  is 


(scrap)  /  (scrap) 


(vr' 


p(scrap)^y  (scrap)  ^ 


^2^(mort)^^(mort)^  _  ^(scrap)  (scrap)  ^ 


(5) 


and  the  ‘mortar’  probability  its  complement  (1  -  scrap).  The  probability  in  Eq.  (5)  depends  only 
on  a^,  and  f  .  The  latter  two  are  computed  and  saved  for  each  excluded 

training  datum,  and  proportionality  constant  a  chosen  subsequently. 


In  short,  after  computing  (-yXscrap)^  ^mort)  (-yXmort)^  excluded  responses  not 

reserved  for  calibration,  the  set  of  excluded  responses  is  changed,  trimmed  feature  covariance 
matrices  recomputed,  and  densities  computed  for  the  set  of  new  unreserved  excluded  responses. 
Again,  is  the  number  of  ‘scrap’  training  data  withheld  as  a  part  of  cross  validation  in 

each  cycle,  n*^™”‘Vith  is  the  similar  number  of  ‘mortar’  training  data  withheld  in  each  cycle. 
Letting  ncyd  be  the  number  of  cycles  of  excluding  some  training  data,  ncyd  x  unreserved 

‘scrap’  training  responses’,  and  ncyd  x  unreserved  ‘mortar’  training  responses  are  cycled 

through  the  excluded  set.  For  a  given  value  of  a  ,  the  ‘scrap’  probabilities  associated  with  these 
values  are  summed  as 

^~(scrap)^_  ^  p  (scrap)  (scrap)  ^ 

j  exluded  not  reserved 

Since  the  number  of  ‘scrap’  responses  that  have  been  thus  excluded  is  known  to  be  ncyd  x 
n(scrap)^ith^  parameter  a}  is  adjusted,  so  that 

=  (7) 

2  2 

Eq.  (6)  is  monotonic  in  a  ,  so  solution  is  unique.  Newton’s  method  started  from  a  =  1,  keeping 

a  from  decreasing  to  less  than  0.1  of  its  previous  value  on  any  iteration,  works  very  well.  Since 
p(scrap)  -  Y  .  Criterion  for  setting  a}  also  sets  the  sum  of  to  the  number  of 

mortar  responses  cycled  through  the  excluded  training  data  set  ncyd  x 
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For  a  prospective  number  of  interval  sub-divisions  naiv,  cross  power  matrices  are  3  ndiv  x  3 
ndiv  square  matrices  requiring  at  least  3  ndiv  vector  outer  products  to  be  summed  (in  Eq.  (1))  to 
avoid  singularity.  This  limits  the  prospective  numbers  of  time  interval  sub  divisions  to 

nd.v  <  min(n(^“"P\  n<“°'"V3  (8) 

but  sub-divisions  near  the  limiting  value  are  expected  to  give  poor  results  due  to  variance  in  the 
cross  power  matrix  estimates. 


In  general  ‘mortar’  identifications  are  made  by  choosing  a  threshold  value  above  which 

responses  are  considered  to  be  due  to  ‘mortars’.  To  obtain  as  few  missed  identifications  as 
possible  for  a  given  level  of  it  is  desirable  to  have  as  few  true  ‘mortar’  responses  with 

P' 


(mort)  jgygj  possible,  that  is,  as  many  with  =  1  -  below  =  1  - 

(mort) 


as  possible.  To  select  a  number  of  interval  sub-divisions  that  will  work  well  with  a 


variety  of  threshold  values,  we  choose  Udiv  minimizing  the  sum 


n 


(miss)  ^  _ 


^  p  (scrap)  (scrap)  ^ 


(9) 


j  mortar,  not  reserved 

If  some  ‘mortar’  response  looks  totally  like  a  ‘scrap’  response  then  would  contribute  1  to 
<n(™‘®9>  in  summation  in  Eq.  (9);  if  it  looks  nothing  like  ‘scrap’  responses  it  would  contribute  0 
to  So,  quantity  is  a  measure  of  how  much  the  ‘mortar’  training  data  look  like 

‘scrap’  responses  under  the  classifier  (choice  of  ndiv)  being  considered.  Eor  each  value  of  Udiv 
tried,  {g  computed  using  values  (yXscrap)  ^j^ich  are  computed  using  cross 

validation. 


Recapping,  1/3  of  the  training  data  is  reserved  for  later  use;  the  remaining  ‘mortar’  and  ‘scrap’ 
data  are  randomly  ordered  within  each  of  these  classes.  A  sequence  of  candidate  ndiv  values  are 
cycled  through  as  an  outer  loop.  Sets  of  excluded  ‘mortar’  and  ‘scrap’  training  data  are  chosen 
starting  with  the  first  on  their  randomly  ordered  lists.  Trimmed  feature  covariance  matrices  are 
computed  excluding  these  and  the  reserved  data.  Quantities  (-y/scrap)^  ^mort)  ^^Xmort)^ 
computed  for  the  excluded  data.  The  sets  of  excluded  responses  are  changed  (moving  down  the 
random  ordered  lists),  trimmed  feature  covariance  matrices  recomputed,  and  densities  computed 
for  the  new  set  of  unreserved  excluded  responses,  based  on  the  non-excluded  unreserved 
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responses.  After  eyeling  essentially  all  of  the  non-reserved  training  data  through  the  exeluded 
sets,  a  is  ehosen  (for  the  eurrent  ndiv)  using  Newton’s  method  to  enforee  eriterion  in  Eq.  (7). 
That  is,  so  that  the  sum  of  ‘serap’  probabilities  over  responses  that  have  been,  in  turn,  exeluded 
from  the  eovarianee  matrix  sums  (Eq.  (1))  and  the  empirieal  density  sum  (Eq.  (2)),  is  equal  to  the 
number  of  ‘serap’  responses  among  these.  Then,  with  this  value  of  a  ,  the  sum  of  estimated 
‘serap’  probabilities  over  the  ‘mortar’  responses  that  have  been  in  turn  exeluded,  is  eomputed 
(Eq.  (9)).  This  whole  sequenee  is  repeated  for  eaeh  eandidate  value  of  Udiv,  and  ndiv  giving  the 
lowest  value  of  Eq.  (9)  is  seleeted. 

Onee  Udiv  has  been  seleeted,  eovarianee  matriees  and  are  reeomputed  using  all  non- 

reserved  training  data  to  eompute  feature  medians  and  MADs  for  ‘mortar’  and  for  ‘serap’ 
responses,  and  in  forming  the  eovarianee  matriees  themselves.  Similarly,  all  non-reserved 
training  data  are  then  used  in  reforming  empirieal  distributions  for  ‘serap’  and  ‘mortar’  elasses 
analogously  to  Eqs.  (2)  and  (3)  but  summed  over  all  the  non-reserved  responses,  omitting  any 
self-response  terms  (i  =  j),  with  the  numbers  of  non-reserved  ‘mortar’  and  ‘serap’  responses 
j^(mort)  j^(scrap)^  replaeing  appropriately.  Then  all  data,  reserved  and  non-reserved,  is 

used  in  eomputing  an  estimated  number  of  serap  responses  analogously  to  Eq.  (6),  and 

(X'  reseleeted,  so  that  the  resulting  is  equal  to  the  total  number  of  serap  responses,  to 

ealibrate  the  resulting  empirieal  distributions  and  probabilities.  In  this  step,  the  inelusion  of  non- 
reserved  data  (whieh  entered  into  the  eovarianee  matriees)  in  the  probability  sums  used  in  the 
final  ealibration,  may  bias  the  resulting  probabilities  somewhat,  but  is  thought  to  be  more  than 
eompensated  for  in  redueed  varianee  in  the  final  value  of  a  obtained.  The  inelusion  of  the  1/3 
of  reserved  data,  that  was  omitted  in  estimation  of  the  feature  eovarianee  matriees,  in  the 
probability  sum  used  in  the  final  ealibration,  lessens  the  effeet  of  any  sueh  bias. 

The  resulting  eovarianee  estimates,  empirieal  probability  distributions  for  and  v™*”^  and 
proportionality  eonstant  a  are  then  used  to  evaluate  the  probability  that  a  response  is  due  to 
‘serap’  through  Eq.  (5)  evaluated  using  the  response’s  feature  veetor,  shifted  and  normalized  as  a 
prospeetive  ‘serap’  response  and  as  a  prospeetive  ‘mortar’  response 
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As  mentioned  earlier,  in  applieation  to  Camp  Sibert  training  data,  the  proeess  is  used  separately 
for  elassifieation  of  responses  into  (a)  mortar/half-round  vs.  serap  elasses,  and  (b)  mortar  vs. 
half-round/serap  elasses.  For  training,  45  mortar,  13  half-round,  and  42  serap  training  data 
polarizability  responses  were  available  from  the  GPO,  SE-1,  SE-2,  and  SW  areas  at  Camp  Sibert. 
This  resulted  into  elasses  with  57  and  42  training  data  for  the  diserimination  problem  (a)  and 
with  44  and  55  training  data  for  the  diserimination  problem  (b). 

Between  1  and  6  data  time  interval  divisions  (ndiv)  were  tried  in  eross  validation.  Eor  the 
mortar/half-round  vs.  serap  problem,  <nmiss>  ranges  from  a  maximum  of  0.0364  for  Udiv  =  1,  to  a 
minimum  of  0.0342  for  Udiv  =  2,  so  Udiv  =  2  was  seleeted.  When  reserved  data  is  ineluded, 
<flmiss>  drops  to  0.0324  for  ndiv  =  2.  These  rates  are  eonsistent  with  two  of  the  training  data 
mortar  or  half-round  responses  being  very  similar  to  serap  responses.  Eor  the  mortar  vs.  half- 
round/serap  problem,  in  eross  validation,  <nmiss>  ranges  from  a  maximum  of  0.0892  for  Udiv  =  3, 
to  a  minimum  of  0.0336  for  Udiv  =  6,  so  Udiv  =  6  was  seleeted.  When  reserved  data  is  ineluded, 
<flmiss>  drops  to  0.0222  for  Udiv  =  6.  The  latter  rate  is  eonsistent  with  one  training  data  mortar 
response  being  very  similar  to  half-round  or  serap  responses. 

Probability  in  Eq.  (5),  or  its  eomplement  (in  unity)  forms  the  basis  for  diserimination  between 
‘mortar’  and  ‘serap’  elasses;  a  threshold  value  may  be  seleeted  and  every  response  with 
Pmort(v*^™”*^)  less  than  the  threshold  is  elassified  as  ‘serap’,  the  rest  is  elassified  as  ‘mortars’. 
Erom  eued  objeet  interrogations,  and  the  SEl  area  survey  530  responses  were  seleeted  as  being 
likely  due  to  induetion  in  metallie  objeets,  172  eued,  and  358  un-eued.  109  (45  eued)  objeets 
have  probability  greater  than  90%  of  being  mortars,  30  (22  eued)  objeets  have  probabilities 
between  0.1  and  90%  of  being  mortars,  and  the  remaining  390  objeets  have  less  than  0.1% 
ehanee  of  being  a  mortar.  Similarly,  159  objeets  have  greater  than  90%  probability  of  being 
either  a  mortar  or  half-shell,  116  objeets  have  probabilities  between  0.1  and  90%  of  being  either 
a  mortar  or  half-shell,  with  the  remaining  254  objeets  having  less  than  a  0.01%  ehanee  of  being 
either  a  mortar  or  half  shell. 
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5.  PERFORMANCE  ASSESSMENT 


We  have  adopted  performance  criteria  commonly  used  for  surveys  at  the  test  sites.  In  addition  to 
these  criteria,  we  provide  target  characteristics  (location,  size,  and  polarizability)  for  detected 
objects  as  follows: 

(a)  X,  y,  z  (depth  below  the  surface)  location  of  object 

(b)  principal  polarizability  responses  vs.  time  (as  in  Figures  2-4) 

(c)  identification  as  a  scrap  or  intact  UXO 

From  200  cued  survey  locations  23  objects  were  excluded  from  the  discrimination  set  because 
they  were  identified  as  multiple  objects,  and  27  objects  were  in  the  training  data  set.  Hence  we 
provided  discrimination  results  for  remaining  150  objects.  The  ground  truth  shows  that  34  of 
them  were  4.2”  mortars.  We  produced  two  priority  dig  lists.  In  the  first  one  the  ‘stop  digging’ 
point  was  when  any  object  past  this  mark  had  a  very  little  probability  of  being  4.2”  mortar.  This 
list  indicated  that  90  objects  can  be  left  in  the  ground  and  60  objects  have  to  be  dug.  Scoring 
results  from  IDA  showed  that  we  correctly  identified  all  4.2”  mortars,  we  didn’t  miss  any,  and 
we  had  26  false  positives  -  we  identified  scrap  as  UXO  26-times.  The  second  priority  dig  list 
had  the  ‘stop  digging’  point  at  90%  probability  of  any  object  being  4.2”  mortar.  In  this  list  only 
43  objects  were  identified  as  ‘need  to  dig’  while  107  could  be  left  safely  in  the  ground.  Scoring 
results  from  IDA  showed  that  if  we  would  use  this  priority  list  we  would  again  correctly 
identified  all  4.2”  mortars,  and  moreover  we  would  reduce  false  positives  to  9.  The  ROC  curve 
for  the  first  priority  dig  list  is  shown  in  Figure  17. 
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Figure  17:  ROC  curve  for  the  cued  targets  priority  dig  list. 


The  SEl  area  discrimination  set  contained  266  objects.  The  ground  truth  indicated  that  56  of 
them  were  single  4.2”  mortars.  Again,  we  produced  two  priority  dig  lists.  The  first  one  with  the 
‘stop  digging’  point  when  any  object  past  this  mark  had  very  little  probability  of  being  4.2” 
mortar,  and  the  second  one  with  the  ‘stop  digging’  point  at  90%  probability  of  any  object  being 
4.2”  mortar.  In  the  first  priority  dig  list  we  indicated  that  75  objects  have  to  be  dug,  while  191 
objects  can  be  left  in  the  ground.  Scoring  results  from  IDA  showed  that  we  identified  correctly 
all  56  4.2”  mortars,  and  19-times  we  identified  scrap  as  4.2”  mortar  (19  false  positives).  The 
ROC  curve  for  this  priority  dig  list  is  shown  in  Figure  18.  The  second  priority  list  classified  63 
objects  as  ‘need  to  dig’  and  203  objects  as  scrap.  Scoring  results  from  IDA  showed  that  in  this 
case  we  again  identified  correctly  all  56  4.2”  mortars,  and  reduced  false  positives  to  7.  The  ROC 
curve  for  the  second  priority  dig  list  is  shown  in  Figure  19. 
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<se1  priofity  diglist  11282007  SC071129.csv>,  320  320  320  320.  bud.  Test.  SE1 


Figure  18:  ROC  curve  for  SEl  area  for  the  first  priority  dig  list. 


<se1  priority  diglist  11282007  SC071129.csv>.  255  255  255  255,  bud,  Test,  SEl 


Figure  19:  ROC  eurve  for  SEl  area  for  the  seeond  priority  dig  list. 
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Estimation  of  Receiver  Operating  Characteristic  (ROC)  Curves 

This  section  describes  how  we  estimated  ROC  curves  without  knowing  the  ground  truth.  This 
approach  is  applicable  in  cases  when  the  ground  truth  is  not  know. 


Given  a  set  of  objects  with  probability  p  of  being  UXO  and  a  threshold  po  for  considering  object 
with  p>po  to  be  identified  as  UXO,  if  p>po  then  all  are  considered  UXO.  However,  on  average  a 
fraction  (1-p)  of  them  were  not  actually  UXO,  so  they  were  false  alarms.  If  po>p  then  all  are 
considered  non-UXO  (scrap),  but  on  average  a  fraction  p  of  them  were  actually  UXO,  so  these 
are  missed  identifications.  Extending  this  to  a  set  of  m  objects  with  probabilities  pi,  i=l,  m  of 
being  UXO,  the  expected  overall  false  alarm  rate  is 


m 


Ii-P, 

pS^Po 


(10) 


and  the  expected  overall  false  negative  rate  is 


m 


Zp, 

P<Po 


(11) 


where  these  rates  are  per  total  number  of  targets.  Given  the  same  set  of  identification 
probabilities,  the  expected  number  of  UXO  is 

-Pi  ’  (12a) 

and  of  non-UXO  is 

yi-Pi+ iyp,-pf  (12b) 


where  the  uncertainties  are  given  by  the  square  root  of  the  sum  of  the  variances  of  the  m  terms  in 
the  sums,  and  are  the  same  for  the  two  sums.  Using  these  to  renormalize  Eqs.  (10)  and  (11),  the 
expected  false  alarm  rate  per  non-UXO  is 


Zl-P: 

P.SPo 


Zi 


Pi 


(13) 


and  the  expected  false  negative  rate  per  UXO  is 
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6.  COST  ASSESSMENT 


6.1  Cost  Analysis 

Our  preferred  mode  of  operation  involves  deteetion  and  eharaeterization/diserimination  at  the 
same  time.  The  advantage  of  this  mode  of  operation  is  that  target  reaequisition  is  not  neeessary 
for  diserimination.  Henee,  this  eliminates  surveying  eosts  and  a  seeond,  eued,  survey  at  the 
expense  of  smaller  daily  eoverage.  We  demonstrated  this  mode  of  operation  in  the  SEl  area.  It 
took  us  9  days  to  eover  the  area  (-0.5  aeres/day).  A  field  erew  eonsists  of  2  people,  so  the  eost 
of  doing  this  kind  of  survey  was  -$23,000.  It  took  us  3  days  to  survey  200  eued  targets  (60-70 
targets/day).  The  eost  was  -$8,000. 

If  EM6I  survey  eosts  $2,500/aore,  and  surveying  of  100  flags  eosts  $5,000  -  $6,000,  the  total 
eost  of  the  diserimination  of  the  same  area  in  deteetion  and  then  eued  mode  would  eost  $26,000  - 
$29,000  +  eost  of  eued  targets  reaequisition.  None  of  these  numbers  inelude 
mobilization/demobilization  eosts.  However,  in  the  traditional  deteetion  and  then  eued  mode 
mobilization/demobilization  is  required  three  times,  while  in  our  approaeh  only  one 
mobilization/demobilization  is  neeessary. 
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